Students’ mathematical justification abilities in analyzing ChatGPT’s answers
##plugins.themes.bootstrap3.article.sidebar##
##plugins.themes.bootstrap3.article.main##
Abstract
This study investigates students' mathematical justification abilities in analyzing answers obtained from one of the Artificial Intelligence (AI) based services, namely ChatGPT. The subjects of this descriptive qualitative study are 13 fourth-semester students of STKIP-PGRI Bandar Lampung. They were asked to utilize ChatGPT to solve two linear equation system questions adopted from Indonesian university entrance tests in 2019. The students were then requested to analyze and justify the responses generated by ChatGPT. The justifications from students were then analyzed to see their abilities and determine whether the answers from ChatGPT were correct. As many as 61.54% of the respondents could understand the instructions of the two questions and provide reviews for the answers generated by ChatGPT. 38.46% of the respondents did not understand the questions' instructions. The results showed that students may have lacked the necessary ability to analyze the AI-generated answers in depth and explain them in detail. This may be due to weak mastery of the basic concepts of the material and a lack of accuracy. The findings call for paying more attention to students' mathematical justification abilities in the current AI-assisted era.
##plugins.themes.bootstrap3.article.details##
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
The author is responsible for acquiring the permission(s) to reproduce any copyrighted figures, tables, data, or text that are being used in the submitted paper. Authors should note that text quotations of more than 250 words from a published or copyrighted work will require grant of permission from the original publisher to reprint. The written permission letter(s) must be submitted together with the manuscript.References
Baidoo-anu, D., & Owusu Ansah, L. (2023). Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Journal of AI, 7(1), 52-62. https://doi.org/10.61969/jai.1337500
Cheung, B. H. H., Lau, G. K. K., Wong, G. T. C., Lee, E. Y. P., Kulkarni, D., Seow, C. S., Wong, R., & Co, M. T.-H. (2023). ChatGPT versus human in generating medical graduate exam multiple choice questions—A multinational prospective study (Hong Kong S.A.R., Singapore, Ireland, and the United Kingdom). PLoS One, 18(8), e0290691. https://doi.org/10.1371/journal.pone.0290691
Farhi, F., Jeljeli, R., Aburezeq, I., Dweikat, F. F., Al-shami, S. A., & Slamene, R. (2023). Analyzing the students' views, concerns, and perceived ethics about chat GPT usage. Computers and Education: Artificial Intelligence, 5, 100180. https://doi.org/10.1016/j.caeai.2023.100180
Fatmanissa, N., Siswono, E., Yuli, T., Lukito, A., & Ismail, I. (2024). Utilización de una tarea de toma de decisiones: justificación matemática de los estudiantes en la resolución colaborativa de problemas [Utilizing decision-making task: Students’ mathematical justification in collaborative problem solving]. Avances de investigación en Educación Matemática, 26, 85-103. https://doi.org/10.35763/aiem26.5341
Fijačko, N., Gosak, L., Štiglic, G., Picard, C. T., & John Douma, M. (2023). Can ChatGPT pass the life support exams without entering the American heart association course? Resuscitation, 185, 109732. https://doi.org/10.1016/j.resuscitation.2023.109732
García-López, I. M., González González, C. S., Ramírez-Montoya, M.-S., & Molina-Espinosa, J.-M. (2025). Challenges of implementing ChatGPT on education: Systematic literature review. International Journal of Educational Research Open, 8, 100401. https://doi.org/10.1016/j.ijedro.2024.100401
Govender, R. (2023). The impact of artificial intelligence and the future of ChatGPT for mathematics teaching and learning in schools and higher education. Pythagoras, 44(1), a787. https://doi.org/10.4102/pythagoras.v44i1.787
Hanna, G., & Jahnke, H. N. (1996). Proof and proving. In A. J. Bishop, K. Clements, C. Keitel, J. Kilpatrick, & C. Laborde (Eds.), International handbook of mathematics education (pp. 877-908). Springer Netherlands. https://doi.org/10.1007/978-94-009-1465-0_24
Hershkowitz, R. (2020). Shape and space: Geometry teaching and learning. In S. Lerman (Ed.), Encyclopedia of mathematics education (pp. 774-779). Springer International Publishing. https://doi.org/10.1007/978-3-030-15789-0_138
Holmes, W., Bialik, M., & Fadel, C. (2023). Artificial intelligence in education. In Data ethics : building trust : how digital technologies can serve humanity (pp. 621-653). Globethics Publications. https://doi.org/10.58863/20.500.12424/4276068
Iqbal, M., Khan, N. U., & Imran, M. (2024). The role of artificial intelligence (AI) in transforming educational practices: Opportunities, challenges, and implications. Qlantic Journal of Social Sciences, 5(2), 348-359. https://doi.org/10.55737/qjss.349319430
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., & Tseng, V. (2023). Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digital Health, 2(2), e0000198. https://doi.org/10.1371/journal.pdig.0000198
Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4), 410. https://doi.org/10.3390/educsci13040410
OECD. (2018). Education at a Glance 2018: OECD Indicators. OECD Publishing. https://doi.org/10.1787/eag-2018-en
OECD. (2022). Trends Shaping Education 2022. OECD Publishing. https://doi.org/10.1787/6ae8771a-en
Ojeda, A. D., Solano-Barliza, A. D., Alvarez, D. O., & Cárcamo, E. B. (2023). Análisis del impacto de la inteligencia artificial ChatGPT en los procesos de enseñanza y aprendizaje en la educación universitaria [Analysis of the impact of artificial intelligence ChatGPT on the teaching and learning processes in university education]. Formación universitaria, 16(6), 61-70. https://doi.org/10.4067/S0718-50062023000600061
Rahman, M. M., & Watanobe, Y. (2023). ChatGPT for education and research: Opportunities, threats, and strategies. Applied Sciences, 13(9), 5783. https://doi.org/10.3390/app13095783
Santos, L., Mata-Pereira, J., da Ponte, J. P., & Oliveira, H. (2022). Teachers’ understanding of generalizing and justifying in a professional development course. Eurasia Journal of Mathematics, Science and Technology Education, 18(1), em2067. https://doi.org/10.29333/ejmste/11488
Sarumaha, Y. A., & Rizkianto, I. (2022). Promoting mathematical justification through realistic mathematics education classroom. Jurnal Pendidikan Matematika (JUPITEK), 5(2), 83-94. https://doi.org/10.30598/jupitekvol5iss2pp83-94
Sukirwan, S., Muhtadi, D., Saleh, H., & Warsito, W. (2020). Profile of students' justifications of mathematical argumentation. Infinity Journal, 9(2), 197-212. https://doi.org/10.22460/infinity.v9i2.p197-212
Vasconcelos, M. A. R., & Santos, R. P. d. (2023). Enhancing STEM learning with ChatGPT and Bing Chat as objects to think with: A case study. Eurasia Journal of Mathematics, Science and Technology Education, 19(7), em2296. https://doi.org/10.29333/ejmste/13313
Wardat, Y., Tashtoush, M. A., AlAli, R., & Jarrah, A. M. (2023). ChatGPT: A revolutionary tool for teaching and learning mathematics. Eurasia Journal of Mathematics, Science and Technology Education, 19(7), em2286. https://doi.org/10.29333/ejmste/13272
Wei, X., Chu, X., Geng, J., Wang, Y., Wang, P., Wang, H., Wang, C., & Lei, L. (2024). Societal impacts of chatbot and mitigation strategies for negative impacts: A large-scale qualitative survey of ChatGPT users. Technology in Society, 77, 102566. https://doi.org/10.1016/j.techsoc.2024.102566
Welding, L. (2023). Half of college students say using AI on schoolwork is cheating or plagiarism. BestColleges. Retrieved from https://www.bestcolleges.com/research/college-students-ai-tools-survey
Xu, X., Su, Y., Zhang, Y., Wu, Y., & Xu, X. (2024). Understanding learners' perceptions of ChatGPT: A thematic analysis of peer interviews among undergraduates and postgraduates in China. Heliyon, 10(4), e26239. https://doi.org/10.1016/j.heliyon.2024.e26239
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 39. https://doi.org/10.1186/s41239-019-0171-0
Zhai, X. (2022). ChatGPT: Artificial Intelligence for Education. Retrieved from https://doi.org/10.13140/RG.2.2.35971.37920